arXiv:1508.06230vl [math.OC] 25 Aug 2015 


Cloud-Based Centralized/Decentralized Multi-Agent Optimization 

with Communication Delays 

Matthew T. Hale,^ Angelia Nedic,* and Magnus Egerstedt^ 


Abstract —We present and analyze a computational hybrid 
architecture for performing multi-agent optimization. The op¬ 
timization problems under consideration have convex objective 
and constraint functions with mild smoothness conditions 
imposed on them. For such problems, we provide a primal- 
dual algorithm implemented in the hybrid architecture, which 
consists of a decentralized network of agents into which cen¬ 
tralized information is occasionally Injected, and we establish 
its convergence properties. To accomplish this, a central cloud 
computer aggregates global information, carries out compu¬ 
tations of the dual variables based on this information, and 
then distributes the updated dual variables to the agents. 
The agents update their (primal) state variables and also 
communicate among themselves with each agent sharing and 
receiving state Information with some number of its neighbors. 
Throughout, communications with the cloud are not assumed 
to be synchronous or instantaneous, and communication delays 
are explicitly accounted for in the modeling and analysis of 
the system. Experimental results are presented to support the 
theoretical developments made. 

1. Introduction 

Algorithms for multi-agent and distributed optimization 
have been considered for a variety of problem formulations in 
part because of the varied collection of application domains 
in which such problems arise. Applications of multi-agent 
optimization can be found in robotics [4], [2], [18], power 
systems [6], sensor networks [20], [8], [17], and communi¬ 
cations [1], [23], [7], [13]. 

These diverse applications lead to optimization problems 
of many different formulations. Correspondingly, algorithms 
have been developed that allow for a broad range of problem 
characteristics. For example, in [15] distributed linear pro¬ 
grams with constraints on network connectivity and memory 
are considered. In [3] the authors consider a distributed 
method for minimizing a sum of convex functions over 
a digraph. The authors of [14] devise an algorithm for 
minimizing a sum of possibly non-differentiable convex 
functions over a time-varying digraph. Problems with time- 
varying graphs, non-differentiable objective functions, and 
noisy communication links are considered in [19], while [12] 
considers similar problems in which nodes in the network are 
expected to fail over time. 

The development of decentralized methods in optimization 
has been motivated in part by the fact that centralized meth- 
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ods may not scale well for very large networks of agents [21, 
Section 1.1]. At the same time, centralized methods can more 
efficiently solve some problems, like the Credit Assignment 
Problem in multi-agent robotics, than decentralized meth¬ 
ods [16, Section 3.1]. Such examples indicate that centralized 
information may be rich in a way that could be useful in 
networks which would otherwise be purely decentralized. In 
adding a centralized component to a decentralized network, 
it seems likely that the centralized component would operate 
slower than the decentralized components. Nevertheless, one 
may ask whether it would be useful to occasionally inject 
global information into a multi-agent network where such 
information would otherwise be absent. 

Towards answering this question, we present here a multi¬ 
agent optimization architecture in which a cloud computer 
is used to occasionally provide centralized information to a 
network of agents solving a nonlinear programming problem. 
This cloud-based optimization architecture was introduced 
in [5], though here we substantially broaden the class of 
problems to be solved and allow for communications delays 
when communicating with the cloud. The cloud carries out 
computations based on information sent to it by agents in the 
network, and intermittently disseminates these results to the 
agents for use in their own computations. At the same time, 
each agent shares its state with some number of neighboring 
agents at each time. In [5], the assumption was made that all 
information in the network was synchronized at each time, so 
that all computations were relying on the same information. 
Here we eliminate this assumption and, as a consequence, 
delays occur which give rise to various kinds of errors. We 
present explicit bounds on these errors in terms of network 
constants and algorithm parameters. 

To solve nonlinear program in a distributed manner across 
many agents, we cast such problems as a variational inequal¬ 
ity and then use Tikhonov regularization, which endows the 
resulting variational inequality with certain properties that 
let us draw from existing convergence results. In particular 
we consider a fixed regularization as was done in [10]. A 
fixed regularization is desirable for multi-agent problems 
because it may be difficult to synchronize the timing of the 
changes in regularization parameters across large networks. 
Accordingly, we use the approach of [10] as a starting 
point, though the problem and approach there are quite 
different from the current paper. In [10], the need for these 
results stems from reducing computation times in Lagrangian 
subproblems associated with a dual optimization scheme 
by allowing inexactness in some computations. Here we 
use a different architecture and different model for delays 



to operate as fast as possible by using the most recent 
information available to the agents. Doing so results in delays 
in communication, and computations that rely on information 
of different ages. The resulting structure of delays will be 
detailed below. 

The rest of the paper is organized as follows. Section [II] 
will cover the background concerning the problem to be 
solved and a centralized method for solving it. Then, Sec¬ 
tion III will cover the cloud architecture and modify the 


centralized solution method to ht with the hybrid cen¬ 
tralized/decentralized system. Section IV will present the 
convergence results and error bounds of the partially decen¬ 
tralized algorithm, and Section jV] will present experimental 
results from an implementation of this algorithm on a team 
of mobile robots. Section [^ will conclude the paper. 


II. Problem Formulation and 
Centralized Solution 

In this section we formulate the problem to be solved. This 
section treats everything globally and the results here will be 
modified later in Section [^ to fit with a hybrid architecture 
described therein. 


A. Variational Inequality Setup 

Consider a multi-agent optimization problem comprised 
of N agents indexed by i £ I N}. Suppose that 

agent i has state Xi G K”' with S N, and let the vector x 
denote the column vector of all states, namely 


/ xi 

X2 


X = 


\ 

G K", 


\ Xn J 


where n = Let each agent have a local objective 

function defined only on its own state, fi{xi). We assume 
that fi : K"* —>■ K is and convex. We also consider a 
global cost that is not necessarily separable, c{x), and assume 
that c : K" —^ M is both and convex as well. We assume 
that agent i knows c and fi, but not for any I that is, 
each agent knows the non-separable cost and its own local 
cost, but not the local cost function of any other agent. We 
assume further that the cloud does not know fi for any i G 1. 

The agents are collectively subject to global inequality 
constraints of the form 

/ 9i{x) \ 

^ ^ 92{x) 

9{x) = . <0, 

V 9mix) J 


for every i, where Xi is non-empty, compact, and convex. 
Letting 

X = Xi X X 2 X * * * X Xj^, 
we encapsulate each set constraint by requiring 
xGX. 


Regarding the class of optimization problems under consid¬ 
eration, we summarize the conditions that we have imposed 
in the following assumption. 

Assumption 1: The set X is non-empty, compact, and 
convex. The functions {ffjjjGJ ^nd c are convex and 
in X, and fi is convex and in Xi for all i G I. ▲ 

For notational convenience, define the function 

N 

fix) ='^fi{Xi) +c{x). 

2=1 


Let Vj;. denote the operator and define the map 
Xf{x) = (v^,(/i(xi)-fc(x)),...,V^„(/Ar(x 7 v)+c(x))y 


We enforce the following assumption on V/. 

Assumption 2: The map V/ is Lipschitz continuous with 
Lipschitz constant Ly. k 

Note that any collection of functions fi and c which are all 
comprise an / that automatically satisfies Assumption]^ 
whenever X is compact (cf. Assumption]^. Concerning the 
constraints g, we have the following assumptions. 

Assumption 3: (Slater’s Condition) There exists a vector 
X G X such that g{x) < 0, i.e., the constraints are strictly 
feasible at x. k 

Assumption 4: The gradient of each constraint, Vpj, j G 
J, is Lipschitz with constant Lj and hence Xg is Lipschitz 
with constant _ 


La = 






2 
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▲ 

A global formulation of the multi-agent optimization prob¬ 
lem under consideration is given by: 

Problem 1: 

minimize f{x) 
subject to g{x) < 0 
X GX. 


0 

Assumption guarantees that Problem ]T] has a solution 
and Assumption [^ guarantees that a dual solution exists 
with no duality gap. Denote an optimal primal-dual pair 
for Problem ]^ by {x,jl). We now define the Lagrangian 
associated with Problem [T] as 


where g : M” -G with m > 1. The constraint functions 
gj : M” —>■ K are assumed to be convex and for all 
j G J := {1,..., m}. In addition to g, each agent’s state is 
also constrained to lie in a given set Xi C i.e., 

Xi G Xi 


L{x,n) = f{x) + p^g{x), 

where a: G X is as defined above and p, is a vector of 
Kuhn-Tucker multipliers in the non-negative orthant of K™, 
denoted M™. By definition, L{-,p) is convex for all p G K™ 
and L{x, •) is concave for every x. Seminal work by Kuhn 










and Tucker [11] showed that optimal primal-dual pairs for 
Problem[T]are saddle points of L. This saddle point condition 
can be expressed concisely as: for all x G X and ^ G K™, 

L(x,/j) < L{x,fi) < L(x,pL). 

The problem of finding Lagrangian saddle points can be 
restated as a variational inequality (e.g., [9, Section 11.1]). 
Let VX and denote the operators and respectively. 
In the variational inequality setting, we wish to hnd a point 
(i, fl) G X X such that 



i Y 

T 

V xL(^Xj fi) 

A F j \ 

A j. 


-Vf^L{x,fl) 


for all {x,fj,) G X X K™. In order to make use of certain 
established results concerning variational inequalities, we 
take two further theoretical steps: first we modify the map 
{XxL — V^L)^ to make it strongly monotone, and second 
we hnd a (non-empty) compact, convex set containing the 
optimal primal-dual vectors. We describe both steps below. 


B. Tikhonov Regularization 
The gradient map 

VxL{x,n) 
-Xfj,L{x,n) 


is monotone, and we use a hxed Tikhonov regularization in 
order to work with a strongly monotone map. This is done 
by regularizing the Lagrangian function as follows: 

= f{x) + ^\\xA+fAgix) - ||lMf, 

where ly > 0 and e > 0, and these values are kept hxed to 
avoid the need to synchronize changes in parameter values 
across many agents. 

Under this regularization, we see that L^AgT) strongly 
convex for all g, G K™ and L^^c(x,-) is strongly con¬ 
cave for all X. These properties imply that VxLu,e 
—are both strongly monotone; hence, the map 
{VxLiy,€ — is also strongly monotone. In addi¬ 

tion, the strongly convex-strongly concave property of 
together with Assumption [T] and Assumptionj^ guarantee the 
existence of a unique optimal primal-dual pair, {xy^^, G 
X X K™. Using the regularized Lagrangian, we now state the 
variational inequality of interest. 

Problem 2: Find the point {xy^^,fiiyA ^ X x such 
that for all (x, /r) G X x K™, 


X 

T 





> 0 . 


0 

We note that determining the solution to the above variational 
inequality is the same as determining the saddle-point of 
the regularized Lagrangian function Ly^^, i.e., {xy^^,flyA G 
X X K™ solves the above variational inequality problem if 
and only if for all (x, p) G X x K™, 


A — puA — (^5 ■ (^) 


C. Bounds on Dual Variables 

We proceed along the lines of [22] and derive a bound on 
(ly^^. Letting x denote a Slater point for the constraints g, 
we define the dual function associated with Ly^^ as 

gyAA ■= rain Ly Ax, A- 

x^X 

Now consider an arbitrary multiplier jl G K™ and let the 
point i G be such that x = minx^x ^iy,e(x, A- '^he 
definition of qy^c we then have 

Qiy,e{A — ^y^e{x,A} — ^iy,e{,Xtyx, A' 

In view of the saddle-point property of {xy^cP^A (‘-f- 
Equation ([T])), it further follows that 

gu^ei^A — ^ly^eix^X, T^A — ^ly^eix, PiyA' 


Using the regularized Lagrangian expression, we have 

quAA < fix) + iAMA + 

< f{A + AMA + ’^Wxf- 

Rearranging terms then gives 

Ttl 

-^AxddjiA < fix) + -j\\xA - Q<yAA, 

i=i 

from which we conclude that for any fl G M™, 

^ fix) + AAA-^^AA 

j {-gAA} 

For any v > Q and e > 0, we certainly have 

quAA = LyAx,A > LqA^A) = fiA+AgiA-^ 

Selecting fi = 0, observe that 


( 2 ) 


Lo.e(i, 0) = fix)- 

Letting = min^j^x fix), the bound in Equation (|^ can 
be simplified to 

^ fix) + AAr-fx 
~ {-gjiA} 

J — ^ l<j<m 

Using that fty^ej > 0 for all j G J and dehning 


Vy 



fix) + A\xr-r^ 

min {-gAA} 

l<j <m 


we see that Vy is non-empty, compact, and convex, and we 
are guaranteed that fiy ,, G Vy. Using Vy, we can now define 
the algorithm used to solve Problem 

Algorithm 1: Given an initial point (a:(0),p(0)) G X X 
Vy, execute the update law 


x{k + 1) = Fix 


x{k) - aXxLyAxik),AA) 


g{k + 1) = AA + tX^L yAxik), p(fc)) 


until some stopping criterion is reached. 


(3) 

(4) 

♦ 

















Here njc[-] and [•] are the projections onto the sets X 
and respectively, with respect to the standard Euclidean 
norm. In the next section we will explicitly reformulate 
Algorithm [T] for the cloud-based multi-agent case. 

III. Cloud Architecture and Hybrid Solution 

We now cover the architecture that will be used to imple¬ 
ment a hybrid form of Algorithm [T] Then we cover how the 
centralized means of solving this problem will be modified 
and deployed on this architecture. 

A. Overview 

Let agent i have neighborhood set Ni containing the 
indices of all agents it is directly coupled to by g and c. 
That is, if the computation of 

dxi 

requires Xj, then j G Ni and agent j sends its state to agent 
i at each time. This structure of communications necessitates 
that i G Nj if and only if j G N,. 

Let the agents share their states with each other at each 
timestep via wireless communication links. In this frame¬ 
work, each agent stores and manipulates a local copy of 
Problem[^onboard and updates its own state within that local 
copy based on computations it performs onboard. Within 
each timestep, agent i computes an updated value of its own 
state, Xi, and then shares the new value of Xi with agent 
j for all j G Ni. In many cases, including in very large 
networks of agents, we expect that the neighborhood set of 
each agent will be a small subset of total collection of agents 
so that \Ni\ < N for all i, where | • | denotes cardinality. 
Computing values of dual variables using Algorithm [T] will 
require all states in the system (see Equation Qx and given 
that |A^i| < N, we see that no agent will be able to perform 
these computations. Lurthermore, there is no assumption that 
the communication graph of the system is connected, nor is it 
even assumed that each agent is coupled to any other agent. 
In such cases, there is not any way to aggregate all states in 
the network onboard a single agent, even after long periods 
of time. 

To fill this gap, we use a cloud computer as was done 
in [5]. The cloud computer is assumed to be capable of 
executing computationally intensive calculations quickly as 
would be the case with a computer cluster or server farm. 
Occasionally every agent sends its state to the cloud and 
after some time each agent receives back an updated dual 
vector that it stores onboard and incorporates into its own 
local calculations of state updates. Because the cloud must 
aggregate all states in the network, it is assumed that there 
are delays in communicating with the cloud and, due to these 
delays. Algorithm [T] will be modified. 

The precise update law used by each agent will be detailed 
below, though for the current discussion it is sufficient to note 
that each agent executes some onboard update law using 
its own state information, information it receives from its 
neighbors, and the most recent dual vector it has received 


from the cloud, regardless of how long ago that dual vector 
was received. Suppose the agents send their states to the 
cloud at some timestep ko and suppose they all have some 
dual vector po onboard which was received just prior to 
sending their states to the cloud (the states sent at time 
ko were not computed using po)- While the agents are 
waiting to receive an updated dual vector, pi, they continue 
to communicate with each other as before and continue to 
use /ig, which is held constant onboard each agent, in their 
computations of state updates. 

Suppose the agents’ states from fcg arrive at the clouc{^ at 
time fcg + Po for some pg G N. With all states received, the 
cloud can compute the next dual update using a rule similar 
to that in Algorithmic Suppose that computing the next dual 
vector takes some number of timesteps gg S N so that pi 
has been computed at time ko + po + Qo- Then the cloud 
sends pi to each agent and it takes rg S N timesteps to 
reach the agents, arriving at time fcg + pg -f qg + rg. Before 
agent i computes a primal update of Xi using pi, it again 
sends its state to the cloud and then uses pi in its subsequent 
computations. Then this process of the agents sharing states 
with their neighbors, receiving a delayed dual update, and 
sending their states to the cloud is repeated. Note that the 
delays pg, gg, and rg are not assumed to be constant but 
instead are associated with fcg and are allowed to vary with 
each communications cycle, i.e., if the agents again send their 
states to the cloud at time fci, there is no need for pi = pg, 
qi = go, or ri = rg. 

B. Update Law Derivation 

To derive the per-agent update law based on the above 
communications scheme, we consider the transfer of infor¬ 
mation through the network starting from the initial point 
(x(0), p(0)) and generalize it to an arbitrary communications 
cycle. 

Define the delay between the agents receiving p(f) and 
p,{t -f 1) as d{t) := pt + qt + rt- When the system is 
initialized, agent i has onboard its own state, a;i(0), the 
primal stepsize a, and the initial dual vector, p(0). The cloud 
is initialized with the same value of p(0) and stepsize r. At 
time k = 0, each agent sends its state to the cloud and then 
the optimization process begins. Upon receiving all agents’ 
states (which comprise the full vector x(0)), the cloud will 
compute p(l) according to 

p(l) = Hp^ [p(0) -G TV^L,.,(a3(0), p(0))]. 

Simultaneously, agent i will use the update la'wj^ 

Xi{k + 1) = Uxi [xi{k) - V^.Li,^^{x{k),p{0))] (5) 

until it receives //(I) from the cloud, and updates of this form 
occur synchronously across the agents. When each agent 

*The agents’ states can anive at the cloud at different times in which 
case the cloud can wait to compute an updated dual vector until it has 
received all agents’ states. Here, we assume all states arrive simultaneously 
for simplicity. 

^For consistency of notation, is written as an argument of the 

full state vector x^{k) here, though ^XiLu,€ will only depend upon the 
states of agents with indices in Ni. 




receives /r(l), it sends the state Xi{d{0)) to the cloud before 
performing any computations with p,(l). After every state of 
the form Xi(d(0)) arrives at the cloud (which is not assumed 
to happen instantly), the cloud will compute /i(2) according 
to 


p(2) = nx)„ 


/r(l) + TV^Lj.,e(a;((i(0)),p(l)) 


(6) 


Equation (|^ shows that Equation ([^ in Algorithm can 
be distributed among the agents in a natural way by having 
each agent update its own state (recall that X = Xi x ■ ■ ■ x 
Xj^). However, Equation reveals that Equation 0 must 
be modified to account for the communication delays present 
in the system. In particular, the arguments of e are no 

longer aligned in time because p(2) is computed using /i(l) 
and values of x that are based on /r(0). 


C. Multi-agent Implementation 

To compactly express this cycle of communication and 
computation, we implement a change in notation. Let p, be 
indexed by the time variable f S N. In this notation, the state 
of each agent will be indexed both over timesteps at which 
the agents compute primal updates and also over which dual 
vector is currently being used in its computations. The time 
index of the agents’ computations will be fc G N and agent 
z’s state will have a superscript to denote the time index of 
the dual variable agent i currently has onboard. The results of 
agent Es state update using p{t) will be denoted x\{k). 
Using this notation, we restate Algorithm to explicitly 
specify the update law for agent i and to account for the 
delays in dual vectors seen above. 

Algorithm 2: Let agent i have initial state x°(0), stepsize 
a, and initial dual vector /i(0). Let the cloud have initial 
multiplier vector p{0) and stepsize r. Execute for each agent 
i, the following two steps: for all fc = 0,..., d{t) — 1, t > 0, 


x\{k + 1) = Tixi x\{k) - aX^,L„^e{x\k),p{t)) 


(7) 


and for all t > 0, 

pit + 1) = n-D, p(t) + {d{t - 1)), p{t)) 


(8) 


until a certain stopping criterion is reached by each agent 
and the cloud. ♦ 


In the setting of Algorithm H we dehne x' -\di-l)) = 
x°{0). 


IV. Convergence Analysis 

We now show that Algorithm “nearly” converges to the 
saddle point of and bound the quantities \\x*{k) — x^^f\\ 
and Wp{t) — Pu,e\\, where x*{k) is obtained by stacking the 
vectors x\{k), i = 1,..., A^. To do so we first establish the 
following lemma concerning convergence when pit) is fixed. 

Lemma 1: Let Assumptions 1-4 hold. Let each agent use 
the primal regularization parameter v. Define the constants 
Ml, = max^gx)„ ||Ai|| and Cf = Lf-\-v-\-MyLg. Enforce that 
the primal stepsize a satisfies 0 < a < ^jCf. Let x\ ^ denote 
the hxed point of Equation ([T]) for agent i with p{t) fixed. 


and define ..., Define also x‘(/c) = 

(x|(fc)^,..., x^(A;)^)^. Eor a fixed dual vector p{t) the 
sequence {x‘(fc)}, fc = 1,..., d{t) satisfies 

\\x\k)-xl\\<ql/^\\x\Q)-x% 

where (0,1) 9 gp := 1 — av{2 — aCf). 

Proof: See [10], Lemma 4.4. ■ 

Next we establish an inequality that will be used to prove 
Theorem [T] below. 

Lemma 2: Let Assumption hold. Then the inequality 

{P 2 - Pl)'^[-g{X 2 ) + g{xi)) > -^\\X 2 -Xi\\'^ 

is satisfied by all xi, X 2 € A" and pi,p 2 € K™, where Mg = 
max^^x ||Vp(x)||. 

Proof: See [10], Lemma 4.1. ■ 

We now state one of the key theoretical results of the 
paper. 

Theorem 1: Let Assumptions [TJQhold. Suppose that each 
agent uses regularization parameter v > 0 and the cloud 
uses regularization parameter e > 0. Let a be bounded as in 
Lemma and let the dual stepsize r be bounded according 
to 

2v 2e \ 

Mj + 2ev' l + e2 J ’ 

Define the constant 


T < mm 


qd := (1 - re)^ +T^ 

which is in the set (0,1) by the definition of t. Then for all 
f G N we have 

+ E {gdMlMlqf^-^'^ + 2T’^MlMlqf-'^l'^') , (9) 

i=l 

where = max^, ygjc ||x — y\\ and qp is as defined in 
Lemma [T] 

Proof: Lor economy of notation we define 
p‘-i=5(x‘-i(d(f-l))). 

We see that 

p{t + 1) = nx)„ [p{t) + - ep{t))] (10) 

and 

x{(]{Xi,^f) . ( 11 ) 

The projection operator [•] is non-expansive and hence 
using Equations and o gives 

(l-re)^\\p{t)-p,,A\''+A\g{S:uA-g'-"f 

- 2t(1 - Te){p{t) - p„^A^{g{x„A “ ( 12 ) 










We now place a bound on the term 
Defining xl~^ = we see that 


Using Equations ( fTTl i, ( |T8| ), ( |T9l ), and ( |20l i in Equation ( [Tfi] ) 
gives 


<\\g(x.^,)-g{xl-^)f + Uxl-^)-g^-^r 

Erom the Lipschitz property of g, we see that 

\\9{Xu,e) - 5(4"^)II < Mg\\Xu,e - xi~^\\ 

and that 

115(4-') - 5‘-'ll < M,\\xi-^ - x^-\d{t - 1))||, 


I4(i+l)-4.ef < 

((l-'re)^+'r^)liM(i)-4.ef + - 2r(l - 

+ ((1 - ref + 4) 

( 21 ) 


Using that 


r < min 


2iy 


2e 


M^ + 2ev’ l + e2 


which now together give 

+ 2M^\\xi-^ - x^-\d{t - l))||p.,, - 4-'ll- (13) 

Examining the term {g,{t) — {g{x„^e) — 5*”^) in 

Equation ( fT2| ), we add and subtract the same term inside 
the second set of parentheses, giving 

(p(f) -/ij.,e)^(5(^>^.e) - 5*-') 

= - 9u,ef {9{Xu,e) - 5(4"') + 5(4"') - 5*-') 

^]y^ll4i.4-5(4-')f+(5(t)-A.4^(5(4-')-5‘-'), 

(14) 

where the last inequality follows from Lemma Multiplying 
Equation ( [T4l l by —2 t( 1 — re) then gives 

- 2t( 1 - re)(/r(f) - - 5*"') 

<-2r(l-re)^|ip(cr..,)-p(x:-i)f+r2||M(f)-A...f 
+ (l-re)2||p(4-')-5‘-'f- (15) 

Substituting Equations ( [T3] l and ( |T5] l into Equation ( [T2] ) 
and combining like terms gives 

\\fi{t + 1) - < ((1 - ref + t2) y{t) - 

+ - 2t(1 - xe)^^ \\g{xu,e) - 5(4-')f 

+ ((1 - re)2 + r^) WgJ-^) - g{x^-\d{t - l))f 

+ 2t^MI\\x,^, - 4-'llll4-' - x^-\d{t - 1))||. (16) 

By Lemma [T] we see that 

ll^‘-i(d(t_i))_4-i|l <5^^(‘-i)/2||^t-i(o)_4-i||. (17) 
Also, by definition of Mg and we have 

||5(i.,,)-5(4-')f < < M^Ml (18) 

while using Mg and Lemma [T] gives 

l!5(4-')-5‘-'f < 

-I)||x‘-1(0) - 4-'f < -1). (19) 

Along the lines of Equation ( [TtI i, we also see that 
||i.,.-4-'llll4-'-4-'(d(f-i))|| <M29f-i)/2. (20) 


we have 

(1 — re)^ + < 1 

and 

-2Til-re) ^ <0. 

IVlg 

Setting = (1 — re)^ + r^, Equation gives 

I4(i+ 1) -4.ef < 

( 22 ) 

Applying Equation ( |22| ) recursively then gives the desired 
result. ■ 

The sum in the statement of Theorem [T] contains terms 
that will eventually become small by virtue of the leading 
term being and i getting large. The interpretation of 
Theorem is that vectors computed by the cloud will 
eventually become close to though the distance between 
them will never become zero. In fact, after a long time this 
distance is dominated by the first few terms of the summation 
in Equation (|^ precisely because the term containing 
goes to zero. We now bound the distance between primal 
vectors and their optima. 

Theorem 2: Let Assumptions 1-4 hold. Then for the se¬ 
quence of primal vectors {x*(c?(f))}tgN generated by Algo¬ 
rithmic] we have 

\\x*{d{t)) - 

along with 

max{0, gj{x*{d{t))} < Mg . 

Proof: Using Lemmas [T] and we see that 

\\x*{d{t)) - x^^eW < ||a;‘(d(f)) - 4ll + Il4 - (23) 

<qm/^M, + ^Mt)-fl,4. (24) 

Using the convexity of each constraint function gj, we see 
that 

9j{x\d{t))) < gj{x,,,e) + Vg{x\d{t)))'^{x\d{t)) - 

< \\'^g{x^^^)\\\\x\d{t)) - x^,4 
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Fig. 2. Team of 8 Khepera III robots executing the cloud-based algorithm. 


Fig. 1. A plot of the final simulation positions, and the final robot 
positions, shown as asterisks, circles, and diamonds, respectively. 


for all j G J, where we have used the bound on \\x\d{t))- 
from Equation ( |2^ . ■ 

The first half of Theorem says that x\ will eventu¬ 
ally become close to with the degree of closeness 

determined in part by the distance between /r(f) and 
The second half makes a similar statement about the extent 
of any constraint violations, namely that the degree of any 
constraint violation depends upon the distance from /r(f) to 
/iy £. Both statements in Theorem say that the length of 
delays between dual updates can be beneficial when it is 
long, though naturally longer delays also require more time 
for convergence. 

While the point e,e) is not necessarily a saddle 
point of the original (unregularized) Lagragian, it is guaran¬ 
teed to be sufficiently close when v and e are small enough; 
an extended discussion of this matter is in [10, Section 3.2]. 
In effect. Algorithm lets the agents approach Xy^^^ which 
itself it not far from an optimal solution to Problem 1 in terms 
of the optimal function value and small feasibility violation 
of the functional constraints. 


As before the total cost used was f{x) = 
The constraints used were 


9{x) 


/ ||a;i - X2\\'^ - 0.6 

ija;! - x^W^ - 1.2 

\\x7 - a;8p - 1.8 
iia;i - xsW^ - 0.4 
V \\xi - - 0.9 


\ 

< 0 , 


and each agent was confined to the box [—1.5,1.5] x 
[-1.0,1.5], giving 


8 

X = ]J[-1.5,1.5] X [-1.0,1.5]. 

i=l 


The constants needed to solve this problem were computed 
(approximately) numerically and are shown in Table [^ 


Symbol 

Value 

Lf 

26.9982 

L, 

4.2426 

My 

8.7430 

M, 

13.5277 

Mx 

V 122 


TABLE I 

Constants associated with Problem[2] 


V. Experimental Results 

Algorithm [^ was simulated for and then run with 8 planar 
agents. We outline the problem and cover the simulation 
results, and then present the experimental results. 

All agents are planar so that Xi G for all i and x G 
The sum of the per-agent objective functions is 




i2:ii 




2 


f 0.2\ 

X2 - 

ij 

+ 

X2, - 

v-o-ej 



^-1.4\ 

2 


F- 0 . 1 \ 

2 

X4 - 

k 1-4 J 

+ 

X 5 - 

F 0 . 5 ; 



Xq 


-0.7 
0.7 

2 I 


-F (ai7,i — 0.5)^ -F X7,2 — l.l + ( 2 ^ 8,1 “F 0.3)^ -F Xg 2- 
The non-separable term in the cost is 


cix) = ^(l|a;i - X4f + ||xi - Xgf + ||a:4 - )• 


The regularization parameters were chosen to be i/ = e = 
0.1, giving Cf = 64.191. The primal and dual stepsizes were 
chosen to be 

a = 0.9 • — = 0.02804 

and 


r = 0.9' 


2v 


2e 


M^ + 2ev' 


l + e2 


= 9.835 • 10 


-4 


All delays had length determined by a random integer 
drawn from a uniform distribution on the integers betwen 
10 and 100, inclusive. Algorithm [^ was run until the agents 
had computed 250,000 state updates, during which time the 
cloud compued 4, 539 dual updates. 

In simulation, the initial total distance between the agents’ 
positions and their regularized optima, ||a;°(0) - Xy^eW, was 
2.6024 and their final total distance, ||a;^®^®(250000) — 
was 0.0843. In addition, after only 25, 000 iterations the total 










































distance of the agents to was 0.1392, indicating that 
fewer steps can be taken while still achieving an acceptable 
ending state. In the dual space, the final distance to the 
regularized optimum was ||/i(4,539) — = 0.0393, 

indicating close convergence in the dual space as well. 

This problem was executed on a team of 8 Khepera III 
robots; these robots are pictured in Figure where they 
are 14 seconds into the experimental run. Position data was 
gathered using an OptiTrack motion capture system and 
the cloud-based algorithm was used to generate position 
waypoints for the agents. The experiment was run until the 
robots and cloud completed 75,000 total updates; this point 
was reached in the middle of a communications cycle, and 
that cycle was allowed to finish, giving 73, 720 total state 
updates by each agent and 1, 340 dual updates by the cloud. 
The final error in the primal space was ||a:^^'*°(73720) — 
= 0.0689 and the final error in the dual space was 
|1 p( 1340) —= 0.0587, indicating close convergence of 
the robots to {x,y^e, A plot of the regularized optima, 
simulation results, and experimental results is shown in 
Figure indicating close agreement among the three sets 
of data plotted there. 

VI. Conclusion 

We presented a hybrid centralized/decentralized algorithm 
for solving multi-agent nonlinear programs with inequality 
constraints. To do this, we used a Tikhonov regulariza¬ 
tion of the problem and a computing regime that spread 
computations across the agents and a cloud computer. The 
architectural model incorporated communications delays in 
the system and approximate convergence of the algorithm 
was proven. Experimental results were provided to show the 
applicability of these results. 
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